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IMAGE PROCESSING METHOD , APPARATUS 
AND MEMORY MEDIUM THEREFOR 

BACKGROUND OF THE INVENTION 
5 Field of the Invention 

The present: invention relates to an image 
processing method, an apparatus and a memory medium 
therefor. 

Related Background Art 
10 For adjusting the color balance of a photographic 

image there are conventionally known two methods, 
namely: 

(1) matching the color balance before picture 
taking; and 

15 (2) correcting the image after picture taking. 

As an example of the method (1), there is known an 
color balance adjusting method utilizing a white 
balance switch for example of a video camera. In this 
method, the white color balance of the taken image is 

20 adjusted by preliminarily taking white paper or the 

like before starting the main picture taking operation. 

The method (2) is widely exercised for example in 
the printing field, but depends on the experience and 
intuition of experts. 

25 On the other hand, with the recent spreading of 

digital cameras and photo scanners, the digitization of 
the photographic image can be easily achieved by the 
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general users. Also the output: equipment represented 
by ink jet printer is progressing toward higher image 
quality and lower cost, that the general users can 
easily print the photograph at home. 
5 However, in case of printing a digitized 

photographic image, the quality of the outputted image 
is still insufficient. 

For example, for printing the RGB signals from an 
input equipment by an output equipment represented by 

10 an ink jet printer through so-called personal computer 
or the like, there has been tried color matching 
between the input equipment and the output equipment 
thorough the C1E XYZ color space by Color Think (trade 
name) in the Apple Mcintosh or by CMS in the Microsoft 

15 Windows (trade name). It is however very difficult to 
execute these adjustments in strict manner, because the 
color reproduction range is naturally different between 
the input equipment and the output equipment, and there 
is also involved a basic difference in the working 

20 principle that the input equipment works on the R, G 

and B light emissions while the output equipment works 
on the C, M, Y and K reflective colors. 

Even if the color matching is achieved between the 
input equipment and the output equipment by overcoming 

25 such difficulties, the image satisfactory for the user 

cannot be obtained merely by strictly reproducing the 
image from the input equipment as a print if such 
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original image from the input equipment is not 
satisfactory. Such situation arises in case of an 
overexposure or an underexposure in the image taking or 
a distorted color balance of the entire image caused by 
5 a "color fog" phenomenon. 

For example, in case of auto picture taking by a 
camera with the AE (automatic exposure) function, there 
results a generally dark image of so-called 
underexposure state if the background of the object is 

10 principally constituted by blue sky, but there results 
an overexposure state if the background is principally 
dark, so that the intended object is not always 
recorded in a best state. 

Also in case of a digital camera, the image is 

15 picked up by a CCD, so that a wavelength region not 
sensitive to the human eye is also contained in the 
image signal. If such signal is processed as a part of 
the RGB signals, the original invisible color becomes 
visible to distort the color balance. Naturally there 

20 is incorporated an infrared cut-off filtering but such 
filtering is not necessarily complete, and the color 
balance cannot be completely corrected within the 
limitation of real-time processing. As a result there 
is caused a "color fog" phenomenon, distorting the 

25 color balance of the entire image. 

Since similar phenomena may occur in the photo 
scanner or the flat bed scanner, the color balance may 
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be distorted in the digitization with such scanner even 
if the original negative or reversal film is in the 
optimum state. 

Therefore, in order to obtain satisfactory output 
5 result it is necessary to correct the input image data 
themselves into the image data of appropriate exposure 
with satisfactory color balance, and, for this purpose, 
there is required a simple method not requiring 
cumbersome operations by the user, with a sufficiently 
10 acceptable processing speed. 

SUMMARY OF THE INVENTION 

In consideration of the foregoing, an object of 
the present invention is to enable color balance 

15 correction of high-definition image data with a simple 
configuration . 

The above-mentioned object can be attained, 
according to the present invention, by an image 
processing method of detecting the luminance of a 

20 highlight point and a shadow point of an original 

image, determining the chromatic! ty of the highlight 
point and the shadow point from plural pixels of the 
luminance, and executing a correction process on the 
original image based on the highlight point, the shadow 

25 point and the chromatic! ty : wherein the correction 

process executes matching of the color solid axis of 
the original image with an axis indicating the 
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luminance and contrast adjustment on a color component 
indicating the luminance of the original image. 

Another object of the present invention is to 
enable judgment of the image state by a very simple 
5 method and optimum correction automatically according 
to such j udgment . 

The above-mentioned object can be attained, 
according to the present invention, by an image 
processing method comprising steps of: 
10 detecting the color solid axis of an original 

image; 

judging the exposure state of the original image 
from the positional relationship between the color 
solid axis and an axis indicating the luminance in a 
15 color space in which the color solid is represented; 
and 

setting an image correcting condition according to 
the result of the j udgment - 

The above-mentioned object can also be attained by 
20 an image processing method for effecting an image 

correction process on an original image according to 
the color distribution thereof, comprising steps of: 

detecting the color solid axis of the original 
image in a predetermined color space; and 
25 controlling the image correction process based on 

the positional relationship of the color solid axis in 
the color space. 
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Still other objects of the present invention, and 
the features thereof, will become fully apparent from 
the following detailed description, which is to be 
taken in conjunction with the attached drawings. 

5 

BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a block diagram of an image processing 
apparatus; 

Fig. 2 is a block diagram of a printer driver; 
10 Fig. 3 is a flow chart showing the control 

sequence of an image correction process unit; 

Fig. 4 is a flow chart showing the sequence of 
preparing a luminance histogram; 

Figs. 5A, 5B and 5C are schematic views of a color 
15 solid indicating the distribution of luminance and hue; 
Fig. 6 is a chart showing a non-linear gamma 
conver s i on f unc t ion ; 

Figs. 7A, 7B, 7C and 7D are charts showing 
features of overexposure and underexposure on a 
20 luminance-saturation plane, wherein Fig. 7A is a chart 
showing the concept of an overexposure; Fig. 7B is a 
chart showing the concept of an underexposure; Fig. 7C 
is a chart showing an example overexposure; and Fig. 7D 
is a chart showing an example of underexposure; 
25 Fig. 8 is a chart showing the manner of 

determining the color solid axis in a third embodiment; 
Fig. 9 is a flow chart showing the control 



- 7 - 

sequence of image correction; and 

Fig. 10 is a flow chart showing the control 
sequence of color fog correction in a second 
embodiment . 

5 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

Now the present invention will be clarified in 
detail by preferred embodiments thereof with reference 
to the attached drawings. 

10 [First Embodiment] 

In the image processing method of the present 
embodiment, the original RGB image data are converted 
into luminance and chromaticity data indicating the 
brightness and the hue for each pixel. In this 

15 operation the pixels are selected in succession, 
skipping a pixel for which the saturation of the 
original image data exceed a predetermined value, and a 
luminance histogram is prepared. Then the luminance 
positions (luminance values) for which the cumulative 

20 numbers from the high and low luminance sides reach 

respectively predetermined values are determined as a 
white position (highlight point) and a black position 
(shadow point). In this operation, in order to improve 
the accuracy of detection of the white and black 

25 positions, there may be adopted a process of excluding 
the high saturation pixels from the luminance 
histogram. 
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Then there are calculated the average value of the 
chromaticity of the pixel data having the luminance 
value identified as the white position and the average 
value of the chromaticity of the pixel data having the 
5 luminance value identified as the black position, and a 
straight line connecting these points in the color 
space is identified as the color solid axis (gray 
line). The inclination of the color solid axis with 
respect to the luminance axis indicates that the color 

10 solid axis which should original indicate the change in 
the luminance only includes a change in the 
chromaticity, namely that there is generated a color 
fog. The color fog is corrected by executing a 
conversion process (rotational matrix calculation) on 

15 all the pixels in such a manner that the color solid 

axis assumes the normal position in the above-mentioned 
color space* In the following the present embodiment 
will be explained in further detail with reference to 
the attached drawings . 

20 [Configuration of image processing system] 

An example of the system in the present embodiment 
is shown in Fig. 1. For example a printer 105 such as 
an ink jet printer and a monitor 106 are connected to a 
host computer 100. The host computer 100 is provided 

25 with an application software 101 such as a word 

processor, a table calculation, an internet blouser 
etc., an operating system (OS) 102, a printer driver 
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103 for preparing print data by processing various 
drawing commands (image drawing command, text drawing 
command, graphics drawing command etc. ) indicating an 
output image issued by the application 101 to the OS 
5 102, and a monitor driver 104 for displaying on the 
monitor 106 by processing various drawing commands 
issued by the application 101. 

The host computer 100 is provided, as the 
hardwares for operating these softwares, with a central 

10 processing unit (CPU) 108, a hard disk driver (HD) 107, 
a random access memory (RAM) 109, a read-only memory 
(ROM) 110 etc. 

As an example of the image processing system shown 
in Fig. 1, there can be conceived a configuration 

15 employing a personal computer compatible with PC-AT of 
IBM and Windows 95 of Microsoft as the operating 
system, installing a desired application for executing 
printing and connecting a monitor and a printer. 
In the host computer 100, based on an image 

20 displayed on the monitor, an application 101 prepares 
output image data utilizing text data classified as a 
text such as a character, graphics data classified as 
graphics and image data classified as a natural image. 
For printing the output image data, the application 101 

25 sends a print output request to the OS 102 and sends 
thereto a group of drawing commands consisting of 
graphics drawing commands for the graphics data and 
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image drawing commands for the image data. In response 
to the output request from the application, the OS 102 
issues the drawing command group to a printer driver 
103 corresponding to the output printer. The printer 
5 driver 103 prepares print data, printable by the 

printer 105, by processing the print request and the 
drawing command group entered from the OS 102 and 
transfers such print data to the printer 105. In case 
the printer 105 is a raster printer, the printer driver 

10 103 executes an image correction process in succession 
on the drawing commands from the OS, then executes 
rasterization into a page memory having a capacity of 
24 bits for the RGB signals, then, after the 
rasterization of all the drawing commands, converts the 

15 content of the page memory into a data format printable 
by the printer 105, for example CMYK data, and 
transfers the converted data to the printer. 

The process executed in the printer driver 103 
will be explained with reference to Fig. 2. 

20 In the printer driver 103, an image correction 

process unit 120 executes an image correction process 
to be explained later, on the color information of the 
image drawing commands contained in the drawing command 
group entered from the OS 102. A correction process 

25 unit 121 for printer rasterizes the drawing commands, 
utilizing the color information subjected to the image 
correction process, thus generating dot image data on 
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the RGB 24-bit page memory. Then it executes masking, 
gamma correction, quantization etc. for each pixel 
according to the color reproducibility of the printer, 
and generates and sends the CMYK data dependent on the 
5 printer characteristics to the printer 105. 
[Image Correction Process Unit] 

In the following there will be explained, with 
reference to Figs . 3 to 7D and 9 , the process executed 
in the image correction process unit 120 on the 
10 original image represented by the image drawing 

commands. The following process is not executed on the 
original image represented by the graphic drawing 
commands or by the text drawing commands . 

The image correction process unit 120 of the 
15 present embodiment executes, as shown in Fig. 3, a 

histogram preparation process (S20) and an image 
correction process (S30) according to the histogram. 
The step S20 prepares a histogram by a process shown in 
Fig. 4, and the highlight point and the shadow point of 
20 the image are determined from thus prepared histogram. 
[Preparation of Luminance Histogram] 
Fig. 4 is a flow chart showing the preparation of 
luminance histogram in the present embodiment. 

Referring to Fig. 4, a step SI initiates the 
25 routine for preparing the luminance histogram of the 
original image. A step S2 determines the ratio of 
selection of the pixels to be used for preparing the 
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luminance histogram from the pixels of the original 
image. In the present embodiment, the luminance 
histogram is prepared from all the pixels (selection 
ratio 1 (or 100%)) in case the image data to be 
processed have 350,000 pixels. In case the entered 
image data have a number of pixels larger than 350,000 
pixels, the pixels are selected (sampled) according to 
the ratio of the total number pixels to 350,000 pixels. 
For example if the input image data are of 3,500,000 
pixels, the selection ratio is 3,500/350 = 10, so that 
the luminance histogram is prepared at a rate of one 
out of 10 pixels (selection ratio 10 or 10%). In the 
present embodiment, the selection ratio is determined 
from the following formula: 

n = int( total pixel number of object image 
data/350,000) 

(however, if n < 1, n is selected as 1, and n 
being a positive number) 

Then a step S3 resets a counter for managing the 
line number or sets it at a predetermined initial 
value, and a step S4 increases the content of the 
counter to a line number of the object line. 

In the present embodiment, the sampling of the 
pixels is executed in the unit of a line. Therefore, 
for a selection ratio n, the line number is divided by 
n, and, if the remainder is 0, the pixels belonging to 
such line are selected for processing ( S5-YES ) . For 
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example, for a selection ratio 10, the line number is 
divided by 10 and, if the remainder is 0, the pixels 
belonging to the line are selected for * processing . 

If the object line is to be skipped and not to be 
5 processed, the sequence returns to the step S4. If the 
object line is to processed, the sequence proceeds to a 
step S6, and the pixels belonging to such line are 
subjected to luminance conversion and chromaticity 
conversion in succession. In the present embodiment, 
10 the luminance conversion and chromaticity conversion 
are executed according to the following equations. 
However such conversion are not limited to these 
equations but can also be achieved by various 
equations : 

15 Y (luminance) = int(0.30R + 0.59G + 0.11B) (Y 

being positive) 

CI (chromaticity) = R - Y 
C2 (chromaticity) = B - Y 

Also in the present embodiment, in order to 
20 improve the accuracy of detection of the white position 
(highlight point) and black position (shadow point), 
the saturation S of an object pixel is calculated by 
the following equation and is compared with a 
predetermined saturation value ( Sconst ) ( S7 ) , and, if 
25 the saturation S is larger, the information of such 

pixel is not reflected in the luminance histogram: 
Saturation S = sqrt(Cl~2 + C2~2) 
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wherein sqrt(x) is a function giving "the square root of 
and x~y means x to a power y. 

Thus, in case S > Sconst, the sequence returns to 
the step S6 and the data of the object pixel are not 
5 reflected in the succeeding process. This is because, 
as will be explained later, the saturation of the white 
position is given by the average saturation of a group 
of pixels of high luminance and such saturation 
indicates an error caused by the color fog, so that the 

10 pixels assumed as of originally high saturation had 
better be eliminated from the calculation of the 
highlight point. The effect of this process will be 
explained by a specific example. For example a yellow 
pixel ( R = G = 255, B = 0) has a luminance Y = 226 and 

15 a saturation S = 227 according to the foregoing 

equations. Thus, this pixel has an extremely high 
luminance and a sufficiently high saturation. Such 
pixel should be considered as originally yellow in most 
cases, rather than considered as generated by yellow 

20 color fogging of an originally colorless pixel. If the 
pixel of such high luminance and high saturation is 
included in the luminance histogram, an error is 
generated in the detected white position. Therefore, 
in the present embodiment, there is determined a 

25 certain saturation (Sconst) and the pixel of which 

saturation exceeds such predetermined saturation is not 
included in the luminance histogram. Such method 
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allows to prevent the error caused by the pixels of 
high saturation in the detected white position, thereby 
improving the accuracy of the white position. 

After such discrimination in the step S7, the 
5 luminance histogram is prepared on the pixels 

satisfying the condition S < Sconst (S8). As the pixel 
data employed in the present invention have 8 bits (256 
density levels) for each of RGB data, the luminance is 
also converted into 256 levels. Consequently the 

10 luminance histogram is obtained by counting the 

frequency of pixels in each of 256 luminance levels. 

Also in the present embodiment, the calculated 
chromaticity values CI, C2 are retained in the 
following manner since they are used for calculating 

15 the average chromaticity of the pixels of each 

luminance level, in the color fog correction to be 
explained later. In structurally arranged variable 
with an index from 0 to 255, the frequency, cumulative 
CI value and cumulative C2 value are set as three 

20 members and the result of calculation for each pixel is 
reflected in a member having the index corresponding to 
the luminance value of the pixel. 

When the process on the object pixel is completed, 
there is discriminated whether all the pixels in the 

25 object line have been processed (S9), and, if any 
unprocessed pixel remains in the object line, the 
sequence returns to the step S6 to repeat the process 
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thereafter. When all the pixels in the object line 
have been processed, a step S10 discriminates whether 
any line remains unprocessed, and, if all the lines 
have been processed, a step Sll terminates the 
5 sequence, but, if any unprocessed line remains, the 

sequence returns to the step S4 and the above-described 
process is repeated with the object line shifted to a 
next line. 

The preparation of the luminance histogram with 
10 the selection of the pixels of the original image data 
in the above-described manner allows to obtain a 
histogram with a minimum necessary number of pixels and 
with consideration of improvement in the accuracy in 
the detection of the white and black positions 
15 afterwards. 

[Determination of White Position (Highlight Point) 
and Black Position (Shadow Point)] 

After the preparation of the luminance histogram, 
it is used for determining the white position 
20 (highlight point) and the black position (shadow 

point). In the present embodiment, the black and white 
positions are defined as points where the cumulative 
luminance frequency reaches 1750 from both ends 
(luminance value 0 and 255) of the luminance histogram 
25 toward the center thereof. 

More specifically, the frequency PY of the pixels 
of a luminance value Y is cumulatively added as P0 + PI 
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+ ... and the luminance value when the cumulative 
frequency reaches 1750 is defined as the luminance 
value YSD of the black position. Then there is 
calculated the average chromaticity of the pixels of 
5 the luminance YSD. As explained in the foregoing, the 
cumulative values of chromaticity for each luminance 
value are calculated at the preparation of the 
luminance histogram (cumulative chromaticity values 
CINtotal, C2Ntotal for pixels of luminance N), so that 

10 the average chromaticity values C1SD, C2SD of the 

pixels with the luminance value YSD corresponding to 
the black position are determined by the following 
equations: 

C1SD = ClYSDtotal/PYSD 

15 C2SD = C2YSDtotal/PYSD 

The white position is similarly determined by 
calculating the cumulative frequency P255 + P254 + ... 
and the luminance value where the cumulative frequency 
exceeds 17 50 is defined as the luminance YHL of the 

20 white position. Then calculated are the average 

chromaticity values C1HL, C2HL of the pixels of the 
luminance YHL in the following manner: 
C1HL = ClYHLtotal/PYHL 
C2HL = C2YHLtotal/PYHL 

25 The above-described calculations allow to 

determine the white position (C1HL, C2HL, YHL) and the 
black position (C1SD, C2SD, YSD) in the (CI, C2, Y) 
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color space . 

In the present embodiment, the cumulative 
frequency is determined from the luminance positions of 
the luminance values 0 and 255, but there may be given 
5 a certain offset such as determination from the 
luminance values 1 and 254. 

As explained above, a step S20 in Fig. 3 
determines the white position (highlight point) and the 
black position (shadow point). 

10 Then a step S30 in Fig. 3 executes an image 

correction process, based on the white and black 
positions determined in the step S20. The image 
correction process in the present embodiment includes 
color fog correction for correcting the color fog in 

15 the original age, exposure correction for correcting 
the luminance contrast for optimizing the exposure of 
the original image, and saturation correction for 
improving the color appearance of the output image. 
Fig. 9 shows the flow of correction process 

20 executed by the image correction process unit 120 in 

the step S30 in F:Lg. 3.* At first a rotation matrix for 
color fog correction is determined, then color balance 
(color fog) is corrected by the rotation matrix, and 
the gamma conversion is executed for the luminance 

25 signal according to the exposure status of the image. 
These steps will be explained in succession. 
[Color Fog Correction] 
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After the above-described determination of the 
white and black positions of the original image in the 
(CI, C2, Y) color space, the color fog correction is 
executed . 

5 If the original image is an ideal image without 

color fog, R = G = B on the colorless axis and the 
calculated chromaticity values for the white and black 
positions should satisfy C1HL = C2HL = C1SD = C2SD. 
However, in the presence of a color fog, the line 

10 connecting (C1HL, C2HL, YHL) and (C1SD, C2SD, YSD ) , 
namely the color solid axis, shows an inclination 
toward the color of such color fog, in proportion to 
the level of the color fog. The color fog correction 
can be achieved by a conversion in such a manner that 

15 the color solid axis coincides with the Y-axis 

( luminance axis ) . The conversion may be achieved by 
rotation and parallel translation of the color solid, 
or by conversion of the coordinate system. 

In the present embodiment, the color solid of the 

20 original image is at first rotated, taking the minimum 
luminance point (lowest point) of the color solid axis 
as the center of rotation, until the color solid axis 
becomes parallel to the Y-axis. Then the coordinate 
system is converted in such a manner that the above- 

25 mentioned minimum luminance point becomes the original 
point of the (CI, C2, Y) space. After such color fog 
correction, the color solid shown in Fig. 5B becomes as 
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shown in Fig. 5C. The conversion provides a result 
having the original point at the minimum luminance 
point and the color solid axis coinciding with the Y- 
axis. Fig. 5A shows a color solid of the image data 
5 having an ideal color distribution without color fog. 
The above-described conversion brings the color solid 
after conversion (Fig. 5C ) close to the ideal color 
solid (Fig. 5A). 

In the rotation for bringing the color solid axis 

10 parallel to Y-axis, the axis and angle of rotation can 
be easily determined from the coordinate values of the 
shadow and highlight points. The method of determining 
the rotation matrix for rotating a solid in a three- 
dimensional space by a desired angle about a desired 

15 axis is already known and will not be explained in 
detail. 

The color fog correction can be corrected, as 
explained in the foregoing, by converting each pixel of 
the original image into pixel data (CI, C2, Y) in the 

20 three-dimensional color space represented by 

chromaticity values and luminance, and applying 
rotation and parallel translation to such pixel data to 
obtain pixel data (CI', C2 1 , Y) for which the color 
solid axis (gray line) connecting the black and white 

25 positions coincides with the Y-axis and the minimum 
luminance point is at the original point of the 
coordinate system . 
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[Adjustment of Contrast and Saturation] 

In the following there will be explained a method, 
for further improving the quality of the image by 
adjustment of contrast and saturation, of judging the 
over/underexposure state of the image in a simple 
manner and accordingly applying gamma correction to the 
luminance signal . 

The contrast adjustment is executed by adjusting 
the luminance of the black position (shadow point) to 
"0" or a value close thereto (for example " 10" ) and 
that of the white position (highlight point) to "255" 
or a value close thereto (for example "245"). 

In the following there will be explained an 
embodiment of judging the over- or underexposure state 
of the image exposure in a simple manner and 
accordingly applying gamma correction to the image 
data. 

At first there are determined points T, T 1 in Fig. 
5B, showing the minimum distance between the color 
solid axis to be corrected and the luminance (Y) axis. 
These points can be easily determined from the 
geometrical relation . 

Then the contrast is so adjusted that the 
luminance component YT 1 of the point T f after the color 
fog correction becomes equal to the luminance component 
YT of the point T. As shown in Fig. 6, a point 
( YT , YT T ) is taken as an inflection point, and, if 
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luminance Y ! after the color fog correction is smaller 
than YT 1 , the luminance is corrected to Y" by a 
function given by a straight line a, and, if it is 
larger than YT ' , the luminance is corrected to Y" by a 
function given by a straight line b. 

Naturally there may be executed a correction 
indicated by a line 12 in Fig. 6, without using T, T 1 . 
If the color solid axis is parallel to the luminance 
axis, the point pair T, T 1 is not uniquely determined, 
and, if T, T' are positioned outside the luminance 
range (0, 255), the point (YT, YT 1 ) cannot be taken as 
the inflection point- In such special case, the 
correction can be made according to the line 12. 

The correction utilizing the closest points T, T 1 
of two axes becomes effective particularly on the image 
involving over- or underexposure. The overexposure is 
induced by a fact that the exposure for the enter image 
is influenced by a lighter area such as sky. In such 
situation the input device represented by a digital 
camera executes suppression of the color of high 
luminance, reducing the saturation of the high 
luminance area. As will be understood from the color 
solid axis of the image subjected to such suppression 
of the color of high luminance, represented on a 
saturation-luminance two-dimensional plane shown in 
Fig. 7A, the pixel closest to gray appears in the high 
luminance area. On the other hand, for an 



- 23 - 



underexposure image, the saturation becomes lower in 
the low luminance area as shown in Fig. 7B, because of 
the suppression of the color of low luminance. 

In the actual image, the luminance axis of the 
color solid assumes a form shown in Fig* 7C for an 
overexposed image, on the luminance-saturation plane. 
On the other hand, an underexposed image assumes a form 
as shown in Fig. 7D. If the actual color solid is 
considered to be displaced from the original color 
solid in the ideal state by the influence of image 
taking conditions or input conditions at the A/D 
conversion, the points T, T 1 are considered to indicate 
the positions of minimum displacement. In the present 
invention such displacement is eliminated to obtain 
appropriate gray in a simple manner, thereby correcting 
the brightness level of the entire image. 

It is naturally possible also to use T solely as 
means for simply judging the over- or underexposure 
state of the image, prepare look-up tables (LUT) for 
the under- and overexposure states and execute the 
gamma adjustment of the luminance signal according to 
the luminance component of thepoint T or T 1 . For 
example the contrast adjustment may be made with a 
curve having an inflection point at (YT, YT 1 ) in Fig. 
6. It is thus possible to simply judge, by the values 
of T, T" , whether the image is over- or under exposed. 
Stated different, the luminance component of the point 
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T f , showing the lowest saturation on the color solid 
axis, is positioned at the high luminance side, the 
image shows a luminance-saturation relationship as 
shown in Fig. 7A, and, if the luminance component of 
the point T f is at the low luminance side, the image 
shows a luminance-saturation relationship as shown in 
Fig. 7B. Therefore, in an image subjected to the 
suppression of the color of high luminance and low 
luminance, the image is considered in an overexposure 
state or in an underexposure state if the luminance 
component of the point T f is respectively at the high 
luminance side or at the low luminance side. 

On the other hand, the saturation can be easily 
adjusted by multiplying the color differences CI, C2 
with saturation correction coefficients. As an 
example, for an increase of the saturation by 20% a 
saturation correcting coefficient of 1.2 is adopted, 
and the saturation can be corrected by: 

CI" = 1.2 x CI 1 

C2" = 1.2 x C2 f 
because the saturation is defined by: 

Saturation = sqrt(Cl~2 + C2~2). 

[Inverse Conversion to RGB Space] 

The corrections in the present embodiment are 
completed as explained above. In this state, each 
pixel of the original image is converted from the (R, 
G, B) color signal data into (CI", C2" , Y" ) color space 
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data, which have to be inserve converted into ( R 1 , G', 
B f ) color signal data according to the following 
equations : 

R' = Y" + C1 H 

5 G f = Y" - (0.3/0.59) x CI" - (0.11/0.59) x C2" 

B' = Y" + C2" 

In this manner there can be obtained RGB data 
which are corrected from the original image in the 
color fog, contrast and saturation. 

10 As explained in the foregoing, the present 

embodiment enables secure correction of the color fog 
with limited load of processing. 

Also in the present embodiment, as the sampling 
condition is selected according to the image data size 

15 of the original image, there can be maintained a 

substantially constant relationship between the entire 
frequency in the histogram and the cumulative 
frequencies for determining the white and black 
positions, regardless of the input image. Consequently 

20 there can be realized satisfactory color fog 
correction. 

In addition, by executing the non-linear gamma 
correction on the entire image so as to maintain the 
luminance at a point where the axis of the color solid 
25 of the image to be corrected is closest to the 
luminance axis, there can be achieved contrast 
correction while maintaining the luminance presumably 
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closest to that of the original image. 

Furthermore it is possible to easily obtain the 
over- or underexposure state of the image, and to apply 
a gamma correction by selecting a table according to 
5 such exposure state. 

The pixel sampling may also be executed in the 
unit of a column instead of the unit of a line. 

[ Second Embodiment ] 

In the following there will be explained a second 

10 embodiment, which takes the level of correction into 
consideration with respect to the first embodiment. 

As explained in the first embodiment, if the axis 
of the color solid determined from the image shows an 
excessively large inclination, the correction of such 

15 axis may result in the deterioration of the image 
quality. Such situation may arise in case, for 
example, the color fog is intentionally caused by the 
use of a color filter, or the image is taken under the 
sunset environment . 

20 In such case, such drawback can be avoided by 

judging that the determined highlight and shadow points 
are erroneous and not executing the correction, or 
suitably adjusting the rotation angle thereby 
attenuating the level of correction. Such judgment 

25 that the highlight and shadow points are erroneous can 
be derived from the direction of the color solid axis. 
Since the color of the color fog can be easily 
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judged from the inclination of the color solid axis, it 
is possible not to execute the color fog correction for 
an image taken for example with a color filter for 
obtaining special effect. 
5 In such situation, the angle between the direction 

vector of the color solid axis and the luminance axis 
is used for judging that the color fog correction 
results in a deterioration of the image, and the 
correction is not executed or is weakened in the level. 

10 For example, if the color solid axis is directed toward 
red and is inclined in excess of a predetermined angle, 
for example 40°, such image is judged to inherently 
contain a color fog. The color fog correction with a 
weakened level can be achieved, for example, by raising 

15 the color solid axis by a predetermined angle, for 
example by 20°, or to a predetermined angle, for 
example an angle of 20° to the Y-axis. The rotation 
matrix for such conversion can be easily determined 
from the rotation axis and the rotation angle. 

20 The level of the above-mentioned correction can be 

manually designated by the user, or set in advance 
according to the magnitude of angle and the direction 
of the color solid axis. In the foregoing description 
the color solid of the image showing an inclination of 

25 40° is raised by rotation by 20°, but the level of such 
correction may be designated by the user. The 
succeeding process is same as in the first embodiment. 
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Also, as the color of the color fog can be easily 
judged from the direction of the color solid axis, it 
is possible, for example, not to execute correction in 
the direction of the color of filter, for an image 
5 taken with a color filter for obtaining a special 

effect. In such case, there is designated a color for 
which the color fog is not corrected, then, a rotation 
axis for conversion is set along the non-corrected 
color on the C1-C2 plane and the color solid is rotated 

10 until a plane containing such rotation axis and the 
color solid axis 1 becomes parallel to the Y-axis. 
Such method allows to correct the color fog only for 
the specified color components. 

Fig. 10 is a flow chart showing a sequence for 

15 image correction by rotating the color solid to a 

separately designated angle in case the color solid 
angle is inclined at least by a predetermined angle* 
At first a step S101 determines the highlight and 
shadow points, and a step S102 discriminates whether 

20 the color solid axis is inclined at least by a 
predetermined angle. If it is less than the 
predetermined angle, steps S104, S105 correct color fog 
by such conversion as to bring the color solid axis to 
the luminance axis, as in the first embodiment. 

25 On the other hand, if the color solid axis is 

inclined by at least the predetermined angle, a step 
S103 determines a rotation matrix for rotating the 



- 29 - 



color solid axis by 20° toward the luminance axis, and 
a step S105 executes color fog correction by rotating 
the color solid with such rotation matrix. In this 
process, the angle 20° employed in the step S103 may be 
altered arbitrarily . 

It is also possible to set threshold values at two 
levels, and, if the inclination of the color solid axis 
is larger than a first threshold value ( for example 
40°), to rotate the color solid axis thereby raising it 
(for example 20°) to a not completely gray level. If 
the inclination is between the first and second 
threshold values (for example 20° ), the conversion by 
rotation is not executed, and, if the inclination is 
less than the second threshold value, the conversion is 
executed in such a manner that the color solid axis 
coincides with the luminance axis. Such method allows 
to avoid defective correction for the image having 
intentional color fog. 

As explained in the foregoing, it is possible to 
set at least two threshold values for correction based 
on the inclination of the axis, namely the direction 
and angle thereof, of the color solid constituted by 
the pixels of the image data, thereby judging whether 
the image correction is to be executed, not to be 
executed or executed with an adjusted level, thus 
excluding the defects of the special case in a very 
simple manner. 



I 
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It is also possible not: to correct the color fog 
in a specified color, since the color of the color fog 
can be easily Judged from the direction of the solor 
solid axis. 
5 [Third Embodiment] 

In the foregoing first to third embodiments, there 
has been explained the color balance correction based 
on the highlight and shadow points of the image. In 
the following there will be explained an embodiment of 
10 the correction on other reference points. 

At first there is determined the average color 
difference AE at the highlight and shadow points by the 
following equation: 

AE = sqrt((ClHL - C1SD)~2 + (C2HL - C2SD)~2) 
15 AE should be zero if the color solid axis is 

parallel to the luminance axis, but becomes larger than 
0 if it is inclined. Fig. 8 shows this relation on an 
E (saturation) - Y (luminance) plane. 

Then several sample luminances are selected 
20 between the highlight and the shadow, and the average 
color difference is determined as shown in Fig. 8, for 
the pixels having a luminance Yn in the image and 
having a saturation smaller than AEn. AEn may be a 
predetermined saturation or the average color 
25 difference limited to the direction of inclination of 
the color solid axis. 

Then a line is determined by the least square 
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method on the average color difference of these several 
points, and the process of the first embodiment is 
executed, taking such line as the color solid axis. 

Otherwise there may be executed non-liner color 
5 balance correction by determining a B-spline curve 

approximating the points having the determined color 
difference and luminance and taking such curve as the 
luminance axis thereby bringing the color difference to 
zero . 

10 In this manner the line constituting the luminance 

axis can be obtain not from the highlight and the 
shadow only but from the pixels uniformly sample from 
the entire image. Thus the color fog correction 
reflecting the characteristics of tue entire image can 

15 be achieved by converting the image in such a manner 
that thus obtained line coincides with the luminance 
axis . 

The present invention has been explained by 
preferred embodiments thereof, but the present 
20 invention is not limited by such embodiment and is 

subjected to various modifications within the scope and 
spirit of the appended claims. 
[Other Embodiment] 

The present invention may be applied to a system 
25 consisting of plural equipment (for example host 

computer, interface devices, reader, printer etc. ) or 
an apparatus consisting of a single equipment (such as 
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a copying machine or a facsimile apparatus). 

Also the objects of the present invention can be 
attained in a case where a memory medium storing 
program codes of sequences shown in Figs. 3, 4, 9 and 
5 10 is supplied to the system or apparatus and a 

computer ( or a CPU or an MPU) of the system or the 
apparatus reads and executes the program codes stored 
in the memory medium. 

In such case the program codes themselves read 
10 from the memory medium realize the functions of the 
aforementioned embodiments, and the memory medium 
storing such program codes constitutes the present 
invention. 

The memory medium storing such program codes can 
15 be, for example, a floppy disk, a hard disk, an optical 
disk, a magnetooptical disk, a CD-ROM, a CD-R, a 
magnetic tape, a non-volatile memory card or a ROM. 

The present invention includes not only the case 
where the computer executes the read program codes to 
20 realize the functions of the aforementioned embodiments 
but also a case where an operating system or the like 
functioning on the computer executes all the actual 
process or apart thereof under the instruction of the 
program codes; thereby realizing the functions of the 
25 aforementioned embodiments. 

The present invention further includes a case 
wherein the program codes read from the memory medium 
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are once stored in a memory provided in a function 
expansion board of the computer or a function expansion 
unit connected to the computer, and a CPU or the like 
provided in the function expansion board or the 
function expansion unit executes all the process or a 
part thereof according to the instructions of such 
program codes, thereby realizing the functions of the 
aforementioned embodiments. 



